Papers with progressive alignment hypothesis
Phased Instruction Fine-Tuning for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to enhance pre-trained language models' ability to follow instructions are limited due to the simultaneous handling of varying instruction complexities. |
| Approach: | They propose a phased instruction fine-tuning method that posits that the transition of a pre-trained language model from simple next-word prediction to sophisticated instruction following is a gradual learning process. |
| Outcome: | The proposed method surpasses the one-off instruction fine-tuning method in win rate and validates the hypothesis of progressive alignment. |